Models for algorithmic teaching

Learning theory focuses almost entirely on the learner and its efficient realization, but neglects other parts of the learning process, most importantly the teacher, which is merely modeled as a passive data source. In this thesis, however, we study models in which the teacher plays the central, active role and which allow us to investigate teaching algorithms. In practice, teaching algorithms occur in the shape of intelligent tutoring systems, computer based interactive systems for teaching students or at least aiding their learning process. But in learning theory so far all teaching models fail to properly describe the intelligent tutoring system scenario. We develop and analyze new teaching models that improve the current ones with respect to this scenario. The most common teaching model at present is based on the notion of teaching dimension. This dimension specifies the minimum number of argument-value pairs (“examples”) needed to describe a given Boolean function (“concept”) among a given class of functions. A set of examples that uniquely describes a concept within a class is called a teaching set. One assumption in the teaching dimension model is that all learners are consistent, that is, their hypothesis always matches all known examples. The teaching dimension of a concept then is the minimal number of examples a teacher has to give in order to make all consistent learning algorithms hypothesize that concept. The teaching dimension thus describes the optimal performance of a teacher for a given target concept and therefore in some sense the teachability of the concept. The average teaching dimension over all concepts in a concept class is considered a measure for the teachability of this class, but this value is not known for many natural classes. We show that the classes of monomials and 1-decision lists over n variables both have an average teaching dimension of O(n). As we demonstrate, a straightforward attempt to analyze intelligent tutoring systems in the teaching dimension model fails, because the model does not capture many real-life aspects of teaching. Teachers cannot benefit from arranging the subject matter suitably; the optimal way of teaching is independent of such crucial properties of the learner as its memory size; and the teacher cannot exploit feedback given by the students. Moreover, the teaching dimension proves to be a counterintuitive measure of teachability. For instance, longer 1-decision lists often have a much smaller teaching dimension than shorter 1-decision lists. In this thesis we identify two reasons for this lack of realism. First, the underlying model of the student is too simple. Second, the performance of the teacher is measured with respect to the worst student, rather than all students. We present two approaches to tackle these shortcomings of the teaching dimension model. In the first approach we develop a general, modular framework for algorithmic teaching that allows the specification of various student models and various kinds of students’ feedback. Within this framework, a student is modeled as an algorithm that maintains a hypothesis and changes it according to the examples received from the teacher. We then use this general framework to compare concrete student models. Students differ with regard to the way they change the hypothesis and also what examples they memorize and for how long. In the simplest model we investigate, the students can have a preference for certain hypotheses. We show that if students prefer simple hypotheses over complex ones, 1decision lists become easier to teach the shorter they are. We also consider students with different hypothesis preferences and show that the teachabilities in all such models can be described by a dimension-like parameter, similar to the teaching dimension. From this it follows that the teacher does not need to pay attention to the order of examples or to the feedback. Next, we investigate a model of students in which they develop their hypothesis in a more restricted way than in the teaching dimension model. We show that then the teacher must take care to arrange the material in the right order and that the teacher can benefit greatly from receiving feedback. We also show that optimal teachers in this model are harder to find than in the teaching dimension model. The complexity of the corresponding decision problem increases from polynomial time to NP-complete for memoryless learners and from NP-complete to PSPACE-complete for learners with perfect memory. In the second approach we modify the teaching dimension model so as to analyze the average student instead of the worst student. This is done by introducing a randomized learning algorithm that incorporates all allowed behaviors. We show that to optimize the performance for the average student, the teacher has to pay attention to the order of the material and also to the students’ feedback. Furthermore, the performance of a teacher varies with the students’ memory size. Using the theory of Markov decision processes, we characterize optimal teachers for several variants of randomized learners. We then focus on the randomized learner that does not memorize examples and also provides no feedback to the teacher. There is no known algorithm to compute the performance of the optimal teacher in this case; we show this problem to be NP-hard and devise an approximation algorithm for it. The performance of arbitrary teachers for these learners is hard to calculate, and it is undecidable whether a teacher is successful at all. We identify a class of teachers that can be handled more easily and whose performance can come arbitrarily close to optimal. Finally we show that in general the teacher that chooses the next example greedily does not approximate the optimal performance by a constant factor.